Learning a model of speaker head nods using gesture corpora

During face-to-face conversation, the head is constantly in motion, especially during speaking turns [2]. These movements are not random; research has identified a number of important functions served by head movements [7] [5] [3] [4]. Head movements provide a range of information in addition to the verbal channel such as nods to show our agreement or shakes to express disbelief. The goal of our work is to build a domain-independent model of speaker’s head movements and use the model to generate head movements for virtual agents. To use the model for interactive virtual agents, it needs to operate in real-time. For this reason, we focus on features that are readily available at the time head movements are generated. In addition, we plan to make the model portable to other systems by using features such as part of speech tags that are easily obtainable even when using different language tools. In this paper, we present a data-driven, automated approach to generate speaker nonverbal behavior, which we demonstrate and evaluate by learning when head nods should occur. Specifically, the approach uses a machine-learning technique (i.e. learning a hidden Markov model [8]) to create a head nod model from annotated corpora of face-to-face human interaction, relying on the linguistic features of the surface text. Figure 1 illustrates the overview of the procedures to learn the model. Once the patterns of when people nod are learned, then it can be used to generate head nods for virtual agents by encoding a new sample with the factors used for learning and feeding it to the model to obtain the most likely head movement.